1. Field of the Invention
The present invention relates to an image processing device, an image processing method, and a recording medium, and particularly to an image recording method for correcting, in image data representing an original image formed by recording an object by photography, at least the color imbalance of the original image caused by the light source when the object was recorded by photography, an image processing device which can make appropriate use of the image processing method, and a recording medium for recording a program which enables a computer to function as the image processing device.
2. Description of the Related Art
The color balance of an image visualized on a photographic film by performing developing processing and the like on the photographic film on which an object has been recorded by photography with a camera or the like (i.e. of an image recorded by photography on photographic film), is affected by the characteristics of the photographic film itself and by the processing conditions during the image processing (these will collectively be referred to below as xe2x80x9cfilm characteristicsxe2x80x9d), and the color balance may become biased compared to the color balance of the object when photographed. For this reason, when an image recorded by photography on a photographic film is recorded on a recording material such as photographic paper or is displayed on a display means such as a display unit, it is sometimes necessary to correct the color balance of the image recorded on the recording material or displayed on the display means (referred to below as the xe2x80x9coutput imagexe2x80x9d), so that the color balance of the output image matches the color balance of the object when photographed (i.e. so that the gray portions of the object at the time of photography are reproduced as gray on the output image).
A known example of a method for correcting the color balance of an output image is one in which, assuming that the color of the pixel corresponding to the highlight (for example, the pixel having the maximum density in a negative image, and the pixel having the minimum density in a positive image) of an image recorded by photography on a photographic film (an original image), and the color of the pixel corresponding to the shadow (for example, the pixel having the minimum density in a negative image, and the pixel having the maximum density in a positive image) of the original image are taken as white and black respectively, the gray balance, which represents the color balance on an original image of the gray portions of the photographed object at the time of photography, is estimated and the color balance of the output image is then corrected using the estimated gray balance as a reference. In this method, the pixel corresponding to the highlight and the pixel corresponding to the shadow are each extracted from the original image, and an axis joining in a straight line the points corresponding to both pixels using, for example, RGB density coordinates is determined as the gray axis representing the gray balance.
However, in the above correction method, when the color of the pixel corresponding to the highlight in an original image is not white, such as when the highlight is the forehead of the face of, for example, a Japanese person in an image in which an electronic flash has been used to photograph the person, it is not possible to determine the gray axis representing the proper gray balance (likewise with the shadow), and the color balance of the output image is biased towards the color which complements the color of the pixel corresponding to the highlight. This is known as xe2x80x9chighlight failurexe2x80x9d. The problem has thus been that while the appearance frequency of original images in which this type of highlight failure occurs easily is relatively high, so that likelihood of obtaining an output image with the proper color balance is low.
Another method is known in which, on the basis of Evan""s theory, an image having a uniform LATD (Light Accumulation Transmission Density) for each component color (e.g. RGB), is regarded as being an image in which the color balance matches the color balance of the photographed object at the time of photography (an image in which the gray balance can be determined), the LATD (Light Accumulation Transmission Density) for each component color (e.g. RGB) of the original image is then measured, and the color balance of the output image is corrected so that the LATD for each component color of the output image is uniform.
However, in the above correction method, when regions which are non-gray and in which the hue is substantially uniform (for example, regions corresponding to a green lawn or a blue sky or sea) occupy relatively large portion of the original image, then correcting the LATD so that the LATD is uniform for each component color of the output image results in the color balance of the output image becoming biased towards the color which complements the color of the above regions This is known as color failure. Because the appearance frequency of original images in which this type of color failure occurs easily is relatively high, the problem has been that likelihood of obtaining an output image with the proper color balance is low, as is the case with the method described above.
Japanese Patent Application Laid-Open (JP-A) No. 9-83825 discloses a method in which image data DRxe2x80x2, DGxe2x80x2, and DBxe2x80x2 relating to low saturation pixels is acquired from image data DR, DG, and DB representing an original image, the shadow points DRs, DGs, and DBs and the highlight points DRh, DGh, and DBh are determined for the respective image data DRxe2x80x2, DGxe2x80x2, and DBxe2x80x2, the image data set (DRxe2x80x3, DGxe2x80x3) is obtained by, for each same values of one of a pair of the image data (DRxe2x80x2, DGxe2x80x2) corresponding to each other for each pixel, averaging the values of the other one of the pair (DRxe2x80x2, DGxe2x80x2) corresponding to the same values, the relationship between the density of the two colors is obtained from the set of the image data (DRxe2x80x3, DGxe2x80x3) and the shadow points (DRs, DGs) and highlight points (DRh, DGh), and, on the basis of the relationship thus obtained, at least one of the image data DR and image data DG undergoes linear conversion across the entire area thereof so as to be made equal with the other one of DR and DG.
In the technology described in the above publication, an attempt is made to alleviate the effects brought about when the image is one in which color failure easily occurs by removing from the data to be calculated the data of high saturation pixels from among the image data representing the original image. In addition to the high saturation pixels, pixels which are adjacent to the high saturation pixels and whose hue is different to the high saturation pixels by a predetermined amount or less are also removed from the data to be calculated. However, even if the above processing is actually carried out on an original image in which color failure is likely to occur, many pixels out of those non-gray and substantially uniformly-hued pixels which occupy a relatively large surface portion of the original image are not removed from the data to be calculated and are left remaining. Accordingly, when the original image is one in which color failure is likely to occur, there has been a problem that the color balance of the output image has not been corrected with sufficient accuracy.
Moreover, it is well-known that the color balance of images obtained by photographing and recording on photographic film scenes which are lit by different types of light sources such as fluorescent lights or tungsten lights (alternative light source scene images) is greatly affected by the fact that the spectral distribution of illuminating light from the alternative light sources is hugely different from the spectral distribution of typical light sources such as daylight and the like, and the color balance becomes biased towards a particular color relative to the original color balance (the color balance when illumination is from a typical light source such as daylight) of the photographed object. The color of the pixels corresponding to the highlight in an alternative light source original scene image is a color other than white (the particular color depends on the type of light source), in the same way as in the above described images prone to highlight failure.
However, in order to obtain an output image with the proper color balance (an output image in which the subject of the photograph is reproduced in the original color balance) from an alternative light source scene image, it is necessary to correct the color balance while regarding the color of the pixel corresponding to the highlight as white (gray). In contrast, in order to obtain an output image with the proper color balance from an image in which highlight failure is liable to occur, it is necessary to correct the color balance in such a way that the color of the pixel corresponding to the highlight is preserved (while regarding the color of the pixel corresponding to the highlight as non-gray). As can thus be understood, the course for properly correcting the color balance is completely different for each case.
In conventional color balance correction, no consideration whatever is given to the above. Accordingly, color balance correction which satisfies both the requirement for accurate correction of color balance bias of an original image caused by the film characteristics regardless of the content of the original image (i.e. of whether color failure is likely to occur in the image, or whether highlight failure is likely to occur in the image), and the requirement for accurate correction of color balance bias of an original image caused by the light source when the photograph was taken has not yet been seen.
Furthermore, the problems encountered in the above conventional color balance correction are common to the problems encountered when correcting the color balance of an original image obtained recording an object by photography using a digital still camera (an original image recorded as image data on an information recording medium by photographing an object). Namely, the color balance of an original image represented by image data recorded on an information recording medium by photographing an object using a digital still camera is not only affected by the characteristics of the photoelectric conversion element of the digital camera so as to become biased relative to the color balance of the object at the time of photography, but is also affected by the light source at the time of photography so as to become biased relative to the original color balance of the object of the photograph.
The present invention has been achieved in consideration of the above and a first object thereof is to provide an image processing device capable of accurately correcting bias in the color balance of an original image caused by the light source at a time when an object was recorded by photography.
A second object of the present invention is to provide an image processing device, an image processing method, and a recording medium capable of accurately correcting both bias in the color balance of an original image formed by recording an object by photography caused by the characteristics of the photographing and recording, and bias in the color balance of an original image formed by recording an object by photography caused by the light source at the time of the photographing and recording.
In order to achieve the first object, the image processing device according to a first aspect of the present invention comprises: calculating means for determining the color of a highlight portion in an original image, and the correlation between the color of a highlight portion in the original image and the color of each element forming the original image, based on image data representing the original image formed by photographing and recording an object; determining means for determining the degree or extent of bias in the color balance of an original image caused by the light source at the time the object was recorded by photography, based on the color of the highlight portion in the original image, and the correlation between the color of the highlight portion in the original image and the color of each element forming the original image determined by the calculating means; and correcting means for correcting the color balance of image data in accordance with the color of the highlight portion, based on the degree of bias in the color balance of an original image caused by the light source which degree is determined by the determining means.
The color balance of an original image formed recording an object by photography (i.e. an image visualized on a photographic film by performing developing processing and the like on a photographic film on which an object has been recorded by photography using a camera - in short, an image recorded by photography on a photographic film, alternatively, an image represented by image data recorded on an information recording medium by photographing an object using a digital still camera - in short, an image recorded by photography using a digital still camera) is biased because of the effects of the light source when an alternative light source is used when an object is recorded by photography. However, the present inventors have realized that, depending on the type of light source, the color of the highlight portion in the original image at this time changes to a color of a particular hue and a high level of color saturation and also that the color of non-highlight portions of an original image change to a color tinged with the above particular hue. Accordingly, the present inventors have achieved the first aspect of the present invention.
On the basis of the above, the calculating means according to the first aspect determines the color of the highlight portion in the original image and also the correlation between the color of the highlight portion in the original image and the color of each element forming the original image (a single pixel may be used or a pixel group comprising a plurality of pixels may be used), on the basis of image data representing an original image (i.e. image data obtained by reading an image recorded by photography on a photographic film, or alternatively, image data recorded on an information recording medium using a digital still cameraxe2x80x94in short, image data representing an image recorded by photography). Note here that the correlation determined may be only that for the elements corresponding to the non-highlight portions of an original image, or may be that for all of the elements corresponding to the entire area of the original image. The determining means then determines the extent of bias in the color balance of an original image caused by the light source when the object has been recorded by photography, on the basis of the color of the highlight portion in the original image and the correlation between the color of the highlight portion in the original image and the color of each element forming the original image.
Note that, as the extent of the bias in the color balance, the determining means may determine the existence or non-existence of bias in the color balance of the original image caused by the light source at the time when an object has been recorded by photography or instead may determine the amount (the degree) of the bias in the color balance. Moreover, specifically, the determination of the extent of bias in the color balance by the determining means may be a determination of the amount (degree) of the bias in the color balance made from, for example, the two parameters of: (a) the color (e.g. the saturation) of the highlight portion in an original image; and (b) the correlation between the color of the highlight portion in an original image and the color (e.g. the hue) of each element forming the original image. Alternatively, the determination by the determining means may entail determining the existence or non-existence of bias in the color balance of an original image caused by the light source when an object has been recorded by photography, on the basis of the color of the highlight portion in the image (specifically, the degree of bias in the hue of the highlight portion for the gray balance, for example), then, for the original image in which the existence of bias in the color balance caused by the light source has been determined, determining the amount of the bias in the color balance of the original image caused by the light source, on the basis of the correlation between the color of the highlight portion in the original image and the color (e.g. the hue) of each element forming the original image.
By using the color of the highlight portion of the original image as well as the correlation between the color of the highlight portion of the original image and the color of each element forming the original image, as described above, the determining means is able to precisely determine the extent or degree of (bias in the color balance of the original image caused by the light source when the object has been recorded by photography. Further, because the correcting means corrects the color balance of the image data to accord with the color of the highlight portion in an original image on the basis of the degree of bias in the color balance caused by the light source, which extent has been determined by the determining means, the bias in the color balance of an original image caused by the light source during photographing and recording can be corrected with a great deal of accuracy.
In order to achieve the second object, the image processing device according to the second aspect of the present invention comprises: normalizing means for estimating the gray balance in an original image to be processed, based on image data of a plurality of original images in which objects have been recorded by photography under uniform conditions, and normalizing the image data of the original image to be processed using the estimated gray balance as a reference; calculating means for determining the color of a highlight portion in the original image to be processed, and the correlation between the color of a highlight portion in the original image to be processed and the color of each element forming the original image to be processed, based on image data of the original image to be processed; determining means for determining the degree of the bias in the color balance of the original image to be processed caused by the light source at the time of photographing and recording the object, based on the color of the highlight portion in the original image to be processed, as well as the correlation between the color of the highlight portion in the original image to be processed and the color of each element forming the original image, which have been determined by the calculating means; and correcting means for correcting the color balance of the image data of the original image to be processed in accordance with the color of the highlight portion, based on the degree of the bias in the color balance caused by the light source, which extent was determined by the determining means.
The normalizing means according to the second aspect estimates the gray balance in an original image to be processed, on the basis of image data from a plurality of original images in which objects have been recorded by photography under uniform conditions (for example, objects photographed with a camera and recorded on the same photographic film, or objects whose image data is recorded with a digital still camera on the same information recording medium, or objects whose image data is recorded by photography with the same digital still camera), and normalizing the image data of the original image to be processed using the estimated gray balance as a reference. By using the image data from a plurality of original images as described above, then, for example, even if the original images to be processed or a portion of the original images from the plurality of original images to be processed are images in which color failure is liable to occur, or in which highlight failure is liable to occur, unevenness in the image content of each original image is averaged out over the entire plurality of original images and the photographic and recording characteristics for the plurality of original images can be accurately estimated.
Note that, in the present invention, the term xe2x80x9cphotographic and recording characteristicsxe2x80x9d means the characteristics representing the relationship between the density (or luminance) for each component color in the object and the density of each color component in the image data, and represents the gray balance which varies in accordance with this relationship. These photographic and recording characteristics are defined in accordance with the characteristics of the film when an image is photographed with a camera and recorded on photographic film, the characteristics of the photoelectric conversion element when image data is recorded on an information recording medium by a digital still camera, and the like.
Because a plurality of original images have been recorded by photography under uniform conditions, the photographic and recording characteristics are the same or similar, and by using image data from the plurality of original images, it is possible to accurately estimate the gray balance in the original image to be processed which is biased in accordance with the photographic and recording characteristics. Moreover, by normalizing the image data of the original image to be processed using the estimated gray balance as a reference, it is possible to accurately correct the bias in the color balance (gray balance) of the original image to be processed caused by the photographic and recording characteristics.
Note that, specifically, the normalizing of the image data of the original image to be processed by the normalizing means preferably entails, for example, extracting each candidate gray pixel which has a high likelihood of being a pixel corresponding to a gray portion of the object from each original image, on the basis of image data of the plurality of original images, and then performing the normalization on the basis of the distribution over predetermined coordinates of the group of candidate gray pixels formed from the candidate gray pixels extracted from each original image. Because, in the above, the group of candidate gray pixels is formed from candidate gray pixels extracted from a plurality of original images, the distribution of the group of candidate gray pixels over predetermined coordinates accurately represents the photographic and recording characteristics for the plurality of original images. Moreover, by using this distribution, it is possible to accurately estimate the gray balance in the original image to be processed, and to accurately correct the bias in the color balance of the original image to be processed caused by the photographic and recording characteristics.
Further, specifically, the extraction of the candidate gray pixels from each original image entails, for example, determining high saturation pixels in the original images using the colors of the highlight and shadow in each original image as a reference, and then performing the extraction by removing the high saturation pixels thus determined. Moreover, it is also possible to determine that pixels, from those pixels located adjacent to high saturation pixels in the image, whose hue (or hue and saturation) is similar to the high saturation pixels are also high saturation pixels. Accordingly, it is possible to remove as high saturation pixels almost all pixels estimated to become the cause of color failure.
Moreover, the normalization of the image data of the original image to be processed based on the distribution over predetermined coordinates of the group of candidate gray pixels is preferably performed by performing a linear approximation on the distribution within the density range of the original image to be processed from out of the distribution over predetermined coordinates of the group of candidate gray pixels, estimating the gray axis representing the gray balance, and normalizing the image data of the original image to be processed using the estimated gray axis as a reference.
Because the density range of each original image is unfixed, the density range taken for all of the plurality of original images is generally spread over a wide density range, and, together with this, the photographic and recording characteristics for the plurality of original images are also spread over a wide density range. Moreover, because the photographic and recording characteristics are determined in accordance with the film characteristics or the photoelectric conversion element characteristics or the like, the photographic and recording characteristics are non-linear. On the other hand, the density range of each original image is clearly smaller than the density range of the photographic and recording characteristics, and the photographic and recording characteristics (gray balance information) which are necessary for each original image when normalization using the normalizing means is carried out (correction of the gray balance) are no more than the density range a portion of the entire density range of the photographic and recording characteristics (the density range of each original image).
Accordingly, as stated above, by performing a linear approximation on the distribution within the density range of the original image to be processed from out of the distribution over predetermined coordinates of the group of candidate gray pixels, and by then estimating the gray axis representing the gray balance in the original image to be processed, the gray balance (gray axis) in the density range necessary for the normalization (gray balance correction) of the original image to be processed can be simply and accurately estimated.
Note, however, that although the bias in the color balance of the original image to be processed caused by the photographic and recording characteristics can be corrected by performing the above normalization process using the normalizing means, because bias in the color balance caused by the light source at the time of photographing and recording is unconnected to the photographic and recording characteristics, then, apart from special cases such as when the entirety of a plurality of images have been recorded by photography using the same alternative light source, even if normalization using the normalizing means is performed on the original image which has been recorded by photography using an alternative light source such as a fluorescent lamp or a tungsten lamp, any bias in the color balance caused by the light source at the time of photographing and recording remains uncorrected.
Because of this, in the second aspect: the calculating means determines, on the basis of the image data of the original image to be processed, the color of the highlight portion in the original image to be processed and the correlation between the color of the highlight portion in the original image to be processed and the color of each element forming the original image to be processed; and the determining means determines, on the basis of the color of the highlight portion in the original image to be processed and the correlation between the color of the highlight portion in the original image to be processed and the color of each element forming the original image to be processed, the bias in the color balance of the original image to be processed caused by the light source at the time an object was recorded by photography. This enables the bias in the color balance of the original image to be processed caused by the light source at the time an object was recorded by photography to be accurately determined, in the same way as in the first aspect of the present invention.
Moreover, because the correcting means corrects the color balance of the image data in accordance with the color of the highlight portion in the original image to be processed on the basis of the bias in the color balance caused by the light source determined by the determining means, the bias in the color balance of the original image caused by the light source at the time of photographing and recording can be accurately corrected. Consequently, according to the second aspect of the present invention, both bias in the color balance caused by the photographic and recording characteristics and bias in the color balance caused by the light source at the time of photographing and recording can be accurately corrected in an original image in which an object has been recorded by photography.
According to a third aspect of the present invention, in the second aspect, the calculating means extracts a highlight portion of the original image to be processed using data of pixels other than high saturation pixels in the original image to be processed determined using the colors of a highlight and shadow of the original image to be processed as references, based on image data before normalization has been performed by the normalizing means, determines the color of the extracted highlight portion, and determines, based on the image data of the original image to be processed after normalization has been performed by the normalizing means, the correlation between the highlight portion in the original image to be processed and the color of each element forming the original image.
When determining the color of the highlight portion in the original image to be processed, it is necessary to remove high saturation pixels which have no connection with the highlight portion in the image being processed before extracting the highlight portion. However, because the color of the highlight portion in an original image recorded by photography using an alternative light source has a specific hue and high saturation color which correspond to the light source, if, for example, high saturation pixels are simply determined and removed using the photographic and recording characteristics for the plurality of original images as a reference, then there is a possibility that the pixels corresponding to the highlight portion in the original image to be processed might also be removed as high saturation pixels.
To counter this, in the third aspect, because the highlight portion in the original image to be processed is extracted and the color of the extracted highlight portion determined using the data of pixels other than the high saturation pixels in the original image to be processed, which data has been determined using the colors of the highlights and shadows of the original image to be processed as references, on the basis of the image data before normalization (i.e. before bias in the color balance caused by the photographic and recording characteristics is corrected on the basis of the image data of the plurality of original images) is performed thereon by the normalizing means, the highlight portion in the original image to be processed can be reliably extracted and the color of the highlight portion accurately detected without the pixels corresponding to the highlight portion being removed as high saturation pixels.
Note here that it is preferable that the image data of the original image to be processed used has already undergone normalization processing by the normalizing means (i.e. the image used has had the bias in the color balance thereof caused by the photographic and recording characteristics corrected) when determining the color of an extracted highlight portion.
Moreover, in the third aspect, because the correlation between the highlight portion in the original image to be processed and the color of each element forming the original image is determined on the basis of the image data of the original image to be processed after normalization has been performed thereon by the normalizing means, namely, is determined on the basis of image data in which bias in the color balance caused by the photographic and recording characteristics has been corrected on the basis of image data of a plurality of original images (in an original image in which an object has been recorded by photography using an alternative light source, the image data represents an image in which the color balance of each portion of an image which includes the highlight portion is biased towards a particular hue after being affected by the light source), the correlation between the color of the highlight portion and the color of each element forming the original image can be accurately determined.
Note that as a fourth aspect of the present invention, all the pixels in the original image to be processed may be used as the elements forming the original image. It is also possible that, out of all of the pixels in the original image to be processed, those candidate gray pixels which have a high likelihood of being pixels corresponding to a gray portion in an object may be used. In particular, when the candidate gray pixels which have a high likelihood of being pixels corresponding to the gray portions of the object are used, then even when pixels affected more by the color of the object than by the color of the light source are present in the original image, the degree of bias in the color balance of the original image to be processed caused by the light source at the time when the object has been recorded by photography can be accurately determined without being affected by the above pixels.
In a fifth aspect of the present invention, the distribution of angles formed by highlight color vectors representing the color of a highlight portion and color vectors representing the color of each element forming an original image may be used as the correlation between the color of the highlight portion and the color of each element forming the original image. In an original image in which an object has been recorded by photography using an alternative light source, the color balance of each portion of the image to be processed which includes a highlight portion is biased towards a specific hue corresponding to the light source used for the photographing and recording. Because of this, the angles formed by highlight color vectors and the color vectors of each element (representing the difference in hue) are small angles for all of the elements. Accordingly, it is possible from the distribution of the angles to accurately determine the degree of color bias in an original image cause by the light source during the photographing and recording of the object.
Note that an example of an angle formed by a highlight color vector and the color vector of each pixel includes the inner product (cosine) of both vectors. Further, an example of a distribution of the angles includes an cumulative histogram of the inner products of the vectors.
In a sixth aspect of the present invention, according to the second aspect of the present invention, the correcting means determines original images formed by photographing and recording similar scenes from the plurality of original images, and corrects the color balance of image data of the original images formed by photographing and recording similar scenes by an identical or similar correction amount.
As stated above, in the sixth aspect, the correcting means determines original images (for example, original images whose highlight portions have substantially the same color presumed to be recorded by photography under the same light source) formed recording by photography similar scenes from a plurality of original images, and corrects the color balance of image data of the original images formed recording by photography similar scenes by an identical or similar correction amount. Therefore, when image data corrected by the correcting means is used as the output of the image (i.e. by recording the image on a recording material, displaying the image on a display means, and the like), the finishing of output images corresponding to original images formed recording by photography similar scenes can be made identical or similar.
A seventh aspect of the present invention is an image processing method comprising the steps of: (a) estimating the gray balance in an original image to be processed based on image data of a plurality of original images formed by photographing and recording an object under uniform conditions, and normalizing image data of the original image to be processed using the estimated gray balance as a reference; (b) obtaining the color of a highlight portion in an original image to be processed, and the correlation between the color of the highlight portion in an original image to be processed and the color of each element forming the original image, based on image data of an original image to be processed; (c) determining a degree of bias in the color balance of the original image to be processed caused by the light source present during the photographing and recording of an object, based on the color of the highlight portion in the original image to be processed, and the correlation between the color of the highlight portion in the original image to be processed and the color of each element forming the original image; and (d) correcting the color balance of image data of the original image to be processed in accordance with the color of the highlight portion, based on the above-determined degree of bias in the color balance caused by the light source. Therefore, in the same way as in the second aspect of the present invention, bias in the color balance caused by the characteristics of the photographing and recording as well as bias in the color balance caused by the light source present during the photographing and recording can both be accurately corrected for original images in which objects have been recorded by photography.
An eighth aspect of the present invention is a recording medium on which is recorded a program for executing by means of a computer a routine including: a first step in which the gray balance in an original image to be processed is estimated based on image data of a plurality of original images formed by photographing and recording an object under uniform conditions, and image data of the original image to be processed is normalized using the estimated gray balance as a reference; a second step in which the color of a highlight portion in an original image to be processed, and the correlation between the color of the highlight portion in the original image to be processed and the color of each element forming the original image is obtained based on image data of the original image to be processed; a third step in which a degree of bias in the color balance of the original image to be processed caused by the light source present during the photographing and recording of the object is determined based on the color of the highlight portion in the original image to be processed, and the correlation between the color of the highlight portion in the original image to be processed and the color of each element forming the original image; and a fourth step in which the color balance of image data of the original image to be processed is corrected in accordance with the color of the highlight portion based on the above-determined degree of bias in the color balance caused by the light source.
As described above, a routine which includes the above first through fourth steps, namely a program enabling a computer to function as the image processing device of the second aspect, is recorded on the recording medium according to an eighth aspect of the present invention. Therefore, when the computer reads and executes the program recorded on the recording medium, in the same way as in the second aspect of the present invention, bias in the color balance caused by the characteristics of the photographing and recording as well as bias in the color balance caused by the light source present during the photographing and recording can both be accurately corrected for original images in which objects have been recorded by photography.